Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cross platform support (e.g. with CMake) #77

Open
wants to merge 36 commits into
base: master
Choose a base branch
from
Open

cross platform support (e.g. with CMake) #77

wants to merge 36 commits into from

Conversation

neurolabusc
Copy link

@neurolabusc neurolabusc commented Feb 27, 2020 •
edited
Loading

This pull request resolves issue.

  1. pigz can be compiled with CMake (@ningfei). Benefits include
  • Easy integration with other projects that use CMake
  • Simple ability to build pigz for preferred zlib variant. Specifically, the CMake files allow users to select the traditional system zlib or the higher performance zlib-ng or CloudFlare zlib.
  • Simple ability to build for the Windows operating system using either MInGW or MSVC.
  • Future users can easily extend the CMake to support different zlib variants. The zlib GitHub repository contains over 1300 forks. This provides a modular mechanism to allow pigz to use new zlib variants that have unique features or target specific architectures.
  1. pigz can now be compiled for Windows (@neurolabusc). While there are several other forks that provide Windows support, this new code is unique in that it supports filenames with non-Latin characters.
  2. To validate this pull request (and future development), we have created a Unix testing suite. We have used this to test many operating systems (Ubuntu 14.04, Ubuntu 18.04, MacOS, etc.) as well as ARM CPUs. Beyond measuring speed, it allows users to detect issues with zlib variants.

Caveats: files created by this tool store non-Latin characters in the FNAME tag using UTF8 (not Windows native UTF16). Names stored in the FNAME tag are interpreted as UTF8 characters. This mimics the behavior of pigz and gzip on Unix. Therefore, files created in one operating system can be extracted to another. None of these tools strictly conforms to rfc1952 which forbids non-Latin characters in FNAME. This probably reflects the fact that the gzip format was designed before UTF8 with its clever backward compatibility. Be aware that 7zip will create scrambled names for non-Latin characters, regardless of whether these were generated by pigz, gzip or 7zip itself.

neurolabusc and others added 30 commits December 11, 2019 10:53
@neurolabusc
Initial commit
875edb4
@neurolabusc
Typo
e45a874
@neurolabusc
CMake (not yet working)
788bce3
@ningfei
Reorganize project structure.
dd48942 
Prepare for Superbuild.
@ningfei
Update .gitignore.
691089a
@ningfei
Fix Superbuild.
2bc38ca
@ningfei
Fix warning due to signed integer in malloc size.
05e63d1 
See: google/zopfli@56c07b9
@neurolabusc
Merge pull request #1 from ningfei/master
c1f8a97 
Fix cmake Superbuild.
@neurolabusc
Updated notes
28dd6a9
@neurolabusc
Option for zlib-ng build
93ef899
@neurolabusc
Support for zlib-ng
b11156c
@neurolabusc
Initial Windows support (madler#62), still need to update CMake
02ef1d7
@neurolabusc
Clean up Windows build
5a9bc91
@neurolabusc
Add support for Intel GZip
5373336
@neurolabusc
Experimental Intel zlib support
f1e4cf9
@neurolabusc
Merge pull request #2 from neurolabusc/windows
cd70fd6 
Windows
@neurolabusc
MSVC WCHAR -> utf8 (madler#62)
8d9e823
@neurolabusc
Merge pull request #3 from neurolabusc/windows
0d0abad 
MSVC WCHAR -> utf8 (madler#62)
@neurolabusc
MSVC modifications
02b01f0
@neurolabusc
Chromium zlib
0b7b407
@neurolabusc
Download zlib and pthreads on demand for MSVC
f60c05b
@neurolabusc
Merge pull request #4 from neurolabusc/windows
ce36332 
Windows
@neurolabusc
MinGW64 support for non-Latin characters (madler#62)
baf468a
@ningfei
Change project name in top CMakelists.
add838c 
Affect installation folder name.
@ningfei
Add pthreads4w support for MSVC.
dbf845e
@ningfei
Cosmetic, fix indent.
54c7a2a
@ningfei
Update pthreads4w branch name.
bd71d41
@ningfei
Use https://github.com/ningfei/zlib.git fior now.
c030953
@ningfei
Fix LDFLAG for MinGW.
7665a2c
@ningfei
Statically link to libwinpthread when using MinGW.
84b918e
@neurolabusc neurolabusc mentioned this pull request Mar 10, 2020
Open
@vielmetti
Copy link

Great to see these changes - let me know anything I can do to help them get tested.

@neurolabusc neurolabusc mentioned this pull request May 12, 2020
Closed
@neurolabusc neurolabusc mentioned this pull request Jul 9, 2020
Closed
@TBBle
Copy link

TBBle commented Jul 21, 2020

I assume this PR matches the binaries at https://github.com/neurolabusc/pigz/releases/tag/v2.4.win? I gave them a try, and I noticed that renaming it to unpigz.exe fails to make it decompress automatically. Probably because the code that grabs the program name does not allow for \ or the .exe tail.

The older 2.3-based Windows builds fixes this like so: kjk@8869ee6

@neurolabusc
Copy link
Author

@TBBle thanks. This sounds like a nice feature that extends this pull requests main focus for providing solid support for the Windows operating system. I am happy to revise this pull request or subsequently make a new pull request. @madler do you have a preference?

@neurolabusc
Copy link
Author

@madler I appreciate this is open source. However, I do think this PR makes it easier to compile pigz on Windows and supports Windows non-latin characters. It also eases building for the accelerated CloudShare zlib and zlib-ng. It would be great having this functionality in the main branch. The stall in having this PR reviewed is causing other PRs to be pulled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

cross platform support (e.g. with CMake)?
4 participants
@neurolabusc @vielmetti @TBBle @ningfei